Abstract Title Page 


Title: Effectiveness of Small-Group Tutoring Interventions for Improving the Mathematical 
Problem-Solving Perfonnance of Third-Grade Students with Mathematics Difficulties: A 
Randomized Experiment 


Authors and Affiliations: 

Asha K. Jitendra, Ph.D (University of Minnesota) 
Danielle N. Dupuis (University of Minnesota) 
Michael C. Rodriguez (University of Minnesota) 


SREE Fall 2012 Conference Abstract Template 



Abstract Body 


Background/Context 

Results of the National Assessment of Educational Progress (NAEP; National Center for 
Education Statistics [NCES], 2011) in mathematics document the persistently low achievement 
of students with disabilities, English language learners, students eligible for free/reduced price 
school lunch, and minority students. Among fourth-graders, for example, 45% of students with 
disabilities contrasted with 15% of students without disabilities scored below the basic level on 
the 2011 NAEP (NCES, 2011). Although the focus has been on improving mathematics 
instruction so that all students meet the high standards as measured by state-administered 
achievement tests, it is crucial that students at risk for mathematics difficulties (MD), who vary 
considerably in ability, achievement, and motivation, develop the necessary mathematical 
knowledge to meet grade level benchmarks. 

One approach that is designed to promote positive mathematics achievement outcomes 
for all students is the use of a multitiered service delivery system, the response to intervention 
(RTI) model (Newman-Gonchar, Clarke, & Gersten, 2009). Typically, RTI mathematics models 
include three levels of support; with Tier 1 comprising high quality mathematics instruction 
provided to all students in general education classrooms by the classroom teacher, Tier 2 
comprising targeted supplemental instruction provided in small groups to at-risk students, and 
Tier 3 consisting of intensive individualized instruction for students who demonstrate inadequate 
response to Tier 2 interventions. The focus of this study is on Tier 2 level of support provided to 
at-risk third-grade students for understanding and solving arithmetic word problems involving 
whole number concepts and operations. 

Arithmetic word problems play a prominent role in elementary school mathematics 
curricula. Word problems serve as “a vehicle for developing students’ general problem-solving 
skills” ... [and] “can promote a thorough and broad understanding of, and proficiency with, 
whole-number arithmetic” (Verschaffel, Greer, & De Corte, 2007, p. 583). However, problem 
solving is challenging for students with MD (e.g., Fuchs, Fuchs, & Prentice, 2004), who 
evidence deficiencies in domain-general abilities such as working memory, language, attentive 
behavior, and simultaneous storage and processing speed of information (Andersson, 2008; 
Andersson & Lyxell, 2007; Fuchs et ah, 2010; Geary, Hoard, Byrd-Craven, Nugent, & Numtee, 
2007; Zheng, Swanson, Marcoulides, 2010). 

Based on findings of a research synthesis related to mathematics interventions for 
struggling students, the What Works Clearinghouse (Gersten et ah, 2009) recommended that Tier 
2 interventions in elementary grades should include “instruction on solving word problems that 
is based on common underlying structures” (p. 6). In addition, this instruction should be “explicit 
and systematic” such as “providing models of proficient problem solving, verbalization of 
thought processes, guided practice, corrective feedback, and frequent cumulative review” (p. 6). 
Further, instructional materials should support student “use of visual representations of 
mathematical ideas” and provide opportunities for developing fluent retrieval of basic arithmetic 
facts” (p. 6). These recommendations are consistent with the findings of several meta-analyses of 
mathematics intervention research for at-risk students and students with learning disabilities that 
have identified explicit instruction and teaching students to use heuristics as enhancing 
computation and word problem solving perfonnance (Baker, Gersten, & Lee, 2002; Gersten, 
Chard, Jayanthi, Baker, Morphy, & Flojo, 2009; National Mathematics Advisory Panel, 2008). 
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Purpose / Objective / Research Question / Focus of Study 

Therefore, the purpose of this study was to compare the effects of delivering a 
supplemental, Tier 2 small-group intervention on the mathematics outcomes of at-risk third- 
grade students randomly assigned to either a schema-based instruction (SBI) group or a control 
group using a standards-based, school-provided curriculum. SBI and standards-based curricula 
are similar in their theoretical underpinnings (i.e., emphasize meaningful learning to develop 
conceptual understanding); however, they differ in terms of their instructional practices. 
Standards-based instruction is characterized by an inquiry based, student-directed approach, 
whereas SBI incorporates an explicit, teacher-mediated approach. Our SBI approach is a 
multicomponent intervention based on schema theories of cognitive psychology, research on 
expert problem solvers, and research regarding effective instructional practices for at-risk 
students. We posed the following research questions. 1. Does SBI improve the mathematical 
word problem-solving performance of third-grade students with mathematics difficulties 
immediately following the intervention (posttest) and 8-weeks after the tennination of the 
intervention (delayed posttest)? 2. Does SBI impact transfer from whole number instruction 
involving word problem solving to overall mathematics achievement for third-grade students 
with mathematics difficulties? 3. Does SBI improve the metacognitive strategy knowledge of 
third-grade students with mathematics difficulties? 

Setting/Participants 

Participants were drawn from 28 classrooms across nine elementary schools in a large 
urban school district in the Midwest. Participants were third-grade students receiving 
mathematics instruction in general education classrooms and considered in need of supplemental 
mathematics support. Mathematics and reading scores on the school-administered Measures of 
Academic Progress (MAP; Northwest Evaluation Association, 2010) from the beginning of 
third-grade were examined to select students who scored at or below the 40 th percentile on the 
mathematics subtest and demonstrated a reading score of at least a beginning second-grade level 
based on school district norms. Because students at or below the 40 th percentile represent a fairly 
heterogeneous population of students, we created a dichotomous variable (l st -25 th percentile vs. 
26-40 th percentile) representing students’ at-risk status to test whether SBI was equally effective 
for students from high and low at-risk subgroups. 

A total of 127 students met the above screening criteria and 125 provided informed 
parent consent. Of this group, 115 (92%) participants completed all pretest and posttest 
assessments. Ten students did not complete the study for various reasons. To determine if there 
were significant differences between the original sample (125) and the final sample (115), we 
conducted a series of chi-square analyses. Results showed no significant differences between the 
students who provided data for the final sample and the 10 students who left the study on 
condition, gender, ethnicity, eligibility for free or reduced priced lunch, English language learner 
status and special education status (all p > .40). Demographic information for participating 
students is presented in Table 1. 

A total of 18 tutors (17 females, 1 male) recruited from the community received training 
and provided all instruction in the study. Fourteen tutors were Caucasian, two were Asian 
American, one was African-American, and one was biracial. The majority of the tutors (n =16) 
had bachelor’s degrees, two had master’s degrees and one tutor had a two-year college degree. 
Two tutors also had undergraduate degrees in mathematics. 
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Intervention 

Tutoring sessions were conducted following core mathematics instruction in various 
locations (e.g., library, cafeteria, hallways, another classroom) in the schools students attended. 
In these sessions, students received 30 min of supplemental mathematics instruction using the 
assigned tutoring program (SBI or control) five times a week for 12 weeks. Students in the 
control group received instruction related to place value, whole number addition and subtraction 
computation and strategies, and word problem solving in the number and operations strand of 
their textbook. We developed an instructional package from Investigations in Number, Data, and 
Space for both tutors and students. The tutor package focused on instructional activities from 
four units ( Trading Stickers; Combining Coins; Collections and Travel Stories; Stories, Tables, 
and Graphs; and How Many Hundreds? How Many Miles?). The student package included 
materials from the Student Activity Book that corresponded to daily lessons within each unit. 

For the SBI tutoring, we used the SBI curriculum designed to provide intensive 
instruction on word problem solving instruction. Given that students with MD struggle with 
whole number combinations (Chong & Siegel, 2008; Geary, 2011) and calculation skills (e.g., 
Jordan et al., 2002), we included a pre-unit that comprised four lessons on addition and 
subtraction number combinations, addition categories, inverse relation between addition and 
subtraction using fact families, and place value concept. The core instructional content of the SBI 
intervention focused on enhancing students’ reasoning skills within the context of solving one- 
step and two-step word problems involving Change, Group, and Compare problem types (see 
Table 2). Tutors used the following four instructional practices of (a) priming the mathematical 
structure of problems, (b) visually mapping infonnation in the problem using schematic 
diagrams, (c) providing explicit instruction on a problem-solving heuristic, and (d) promoting 
metacognition of strategy knowledge. Tutors explicitly modeled and scaffolded instruction by 
gradually shifting responsibility for problem solving to the students. 

Research Design 

We conducted a randomized control trial with a pretest-intervention-posttest-retention 
test design. Students within the same classroom, identified as having MD, were randomly 
assigned to either schema-based instruction (SBI) or the control group. Then, students within 
each condition were randomly assigned to instructional groups. Finally, we randomly assigned 
tutors to the instructional groups. Classroom teachers administered the MAP tests in fall and 
spring of the academic year as part of the district-wide evaluation program. We collected pretest 
data on word problem solving (WPS) and metacognitive strategy knowledge two weeks before 
the intervention started. Posttest data on the same variables were collected at the end of the 
intervention as well as eight weeks later on the WPS test to measure retention of problem solving 
skills. Research assistants and tutors were trained to administer these measures in small-group 
arrangements. We gathered tutoring implementation data via observations. 

Data Analysis 

We fitted a two-way analysis of covariance (ANCOVA) model to each outcome of 
interest, where the treatment variable and the at-risk status variable were the effects of primary 
interest and the corresponding pretest measure and demographic variables (gender, ethnicity, and 
eligibility for free or reduced priced lunch [FRL]) were included as covariates. At-risk status was 
coded so that students between the 26 th -40 th percentiles = 1, and students between the 1 st -25 th 
percentiles = 0. Gender was coded so that males = 0 and females = 1, ethnicity was coded so that 
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minority students (African American, Hispanic, Asian American) = 0 and Caucasian students = 
1, and FRL was coded so that students eligible for free or reduced priced lunch = 1 and students 
not eligible = 0. We calculated effect sizes by dividing the covariate adjusted mean differences 
on the posttests by the square root of the mean square error (Glass, McGaw, & Smith, 1981). 

Results 

On the WPS posttest, results indicated a statistically significant treatment effect, F( 1, 
104) = 6.66, p = .011. As hypothesized, SBI students outperfonned control students (cl = 0.48). 
Results also indicated a statistically significant effect for at-risk status, F( 1, 104) = 4.75 , p = .03, 
with low at-risk students outperfonning high at-risk students ( d = 0.45). The interaction between 
the treatment effect and students’ at-risk status was not significant, F( 1, 104) = 0.88, p = .35, 
indicating that SBI was effective regardless of students’ at-risk status. On the WPS delayed 
posttest, the treatment effect was not statistically significant, F( \ , 103) = 1 .09, p = .30. Likewise, 
there was no significant effect for students’ at-risk status, F( 1, 103) = 2.48, p = .12, or the 
interaction between the treatment effect and students’ at-risk status, F(\ , 103) = 0.20. p = .66. 

For the MAP mathematics achievement test, there was a statistically significant treatment 
effect, F( 1, 101) = 4.29, p = .041. As hypothesized, SBI students outperformed control students 
(d = 0.40). Results showed no significant effect for students’ at-risk status, F{ 1, 101) = 0.00 1 , p = 
.98 or the interaction between the treatment effect and students’ at-risk status, F( 1, 101) = 0.32 ,p 
= .57, the latter indicating that SBI was equally effective for students regardless of their at-risk 
status. For metacognitive strategy knowledge, results indicated a statistically significant 
treatment effect, F{ 1, 100) = 12.80, p = .001. SBI students outperformed control students (<7 = 
0.68). The effect of students’ at-risk status was non-significant, F{ 1, 100) = 3.18,/? = .08 as was 
the interaction between the treatment effect and students’ at-risk status, TfiT, 100) = 0.16, p = 
.693. 

Five research assistants observed and audiotaped a total of 54 tutoring sessions in both 
conditions during the 12- week tutoring intervention. Tutor fidelity of implementation was 
evaluated using a 3-point scale (1 = low adherence, 3 = high adherence) for each item on the 
observation forms (SBI and control). SBI tutors averaged 2.93 (SD = 0.16) and control tutors 
averaged 2.79 (SD = 0.25). A second rater listened to the audiotape and rated one randomly 
selected audiotape of a tutor. Inter-rater agreement calculated using kappa ( K) averaged .98 for 
SBI and .78 for the control condition. 

Conclusions 

The present research assessed the efficacy of two tutoring protocols for improving the 
mathematics outcomes of at-risk third-grade students. Results indicated that students in the SBI 
group outperfonned students in the control group on word problem solving performance after 30 
hours of problem-solving experience, but the problem-solving performance was not maintained 8 
weeks following the tutoring intervention. However, the SBI condition led to superior 
perfonnance on a standardized mathematics achievement test compared with the control group. 
With respect to metacognitive strategy knowledge, the SBI condition outscored the control 
condition. Further, results suggested that SBI was equally effective for students from various at- 
risk subgroups. These findings demonstrate the efficacy of SBI tutoring conducted by trained 
tutors. 
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Appendix B. Tables and Figures 

Table 1. Student Descriptive Statistics by Condition 



- 

Control (n = 58) 
N % 

SBI (n 
N 

= 57) 
% 

r/F 

df 

P 

Gender 

Male 

27 

46.5 

28 

49.1 

0.08 

1 

.783 


Female 

31 

53.5 

29 

50.9 




Ethnicity 

Asian/Pacific Islander 

3 

5.2 

2 

3.5 

4.64 

5 

.462 


Black 

18 

31.0 

11 

19.3 





Hispanic 

16 

27.6 

20 

35.1 





American Indian 

2 

3.5 

5 

8.8 





White 

19 

31.0 

19 

33.3 




Eligible for free/reduced priced lunch 

39 

67.2 

38 

70.4 

0.13 

1 

.721 

English language learner 

17 

29.3 

17 

31.5 

0.06 

1 

.803 

Special education status 

8 

13.8 

5 

9.3 

0.56 

1 

.454 

Age (in months): M (SD) 

100.90 (4.02) 

100.91 (3.84) 

0.00 

1, 110 

.988 


Note: SBI = schema-based instruction 
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Table 2 

Solving One-Step and Two-Step Problems Involving Change, Group, and Compare Schemata 


Problem Representation 

Change: Jose and his father have gathered 
10 pounds of wood from a sheep. So far, 
some of the wool has been used to make a 
sweater. Now there are 5 pounds of wool 
left. How many pounds of wool have been 
used? 



Beginning Ending 


Problem Solution 3 


Number sentence: 10 - ? = 5 (We can solve this 
problem by thinking about the fact or number 
families. We know the greater number in this 
subtraction problem. So, we can subtract the lesser 
number from the greater number [10-5 =?] to 
solve for ?. Alternatively, we can think addition to 
solve this problem.) 

Answer: 5 pounds of wool have been used. 


Group: Farmer Jake has 88 animals on his 
farm. He only has horses and goats. There 
are 49 horses on the fann. How many goats 
are on the farm? 


Horses 


Goats 


Horses 
and Goats 

49 

+ 

? 

— 

88 


Small Groups or Parts Large Group or Whole 


Number sentence: 49 + ? = 88 (We can 
solve this problem by thinking about how 
addition and subtraction are related. We can 
undo the addition problem, write a subtraction 
problem: 88- 49 = ?, and solve for ?) 

Answer: There are 39 goats on the farm. 


Compare: A redwood tree can grow to be 
85 meters tall. A Douglas fir can grow to 
be 15 meters taller. How tall can the 
Douglas fir grow? 



Bigger 


Smaller 


Difference 


Number sentence: ? - 85 = 15 (We can 
solve this problem by thinking about how 
addition and subtraction are related. We don’t 
know the greater number in this subtraction 
problem. So, we can undo the subtraction 
problem, write an addition problem, 85 + 15 = ?, 
and solve for ?) 


Answer: The Douglas fir can grow 100 

meters. 


b Two-step problem: Alesha has $15. Arlen 
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has $6 less than Alesha. How much more 
money does Arlen need to buy a theme 
park admission ticket that costs $20. 

Main problem: Change 


+$? 


S9 


B 


$20 


Number sentence: $9 + $? = $20 (We can solve 
this problem by asking ourselves 9 + ? = 20, or we 
can undo the addition problem, write a subtraction 
problem [$20 - $9 = $?], and solve for ?) 

Final Answer: Arlen needs $11 more to 
buy a theme park admission ticket. 


Helper problem: Compare 
Alesha Arlen 


$15 


$9 


B 


S 


Number sentence: 15 - HA = 6 (We can solve this 
problem by thinking about the fact or number 
families. We know the greater number [15] in this 
subtraction problem. So, we can subtract the lesser 
number from the greater number [15-6 = HA] to 
solve for HA.) 


Helper Answer (HA): Arlen has $9. 


Note. “Students also were encouraged to use previously learned solution strategies (see 
Verschaffel et al., 2007) to solve the number sentence; b Diagrams in the two-step problem are 
faded versions. 
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Table 3 


Means and Standard Deviations for Measures by Condition and At-Risk Status 


Condition 


Control 



SBI 


Mean SD 

N 

Mean 

SD 

N 


WPS 








Pretest 








High at-risk a 

13.76 

5.50 

27 


13.45 

5.32 

30 

Low at-risk b 

15.79 

5.35 

31 


18.44 

6.44 

24 

Combined at-risk 

14.84 

5.47 

58 


15.46 

6.38 

57 

Posttest 








High at-risk a 

21.21 

6.68 

27 


24.58 

4.91 

30 

Low at-risk b 

25.52 

7.09 

31 


28.60 

5.76 

24 

Combined at-risk 

23.51 

7.18 

58 


26.39 

5.56 

57 

Adjusted Posttest 








High at-risk a 

22.31 




25.85 



Low at-risk b 

25.59 




27.23 



Combined at-risk 

23.95 




26.54 



Delayed Posttest 








High at-risk a 

22.34 

7.17 

31 


22.94 

6.90 

30 

Low at-risk b 

25.05 

6.58 

30 


28.26 

7.05 

24 

Combined at-risk 

23.77 

6.94 

57 


25.32 

7.31 

57 

Adjusted Delayed Posttest 








High at-risk a 

22.99 




23.72 



Low at-risk b 

24.49 




26.30 



Combined at-risk 

23.74 




25.01 



MAP Mathematics 








Pretest 








High at-risk a 


179.00 

3.22 

27 

177.77 

4.77 

30 

Low at-risk b 


186.35 

1.52 

31 

186.67 

1.69 

24 

Combined at-risk 


182.93 

4.43 

58 

181.72 

5.80 

54 

Posttest 








High at-risk a 


194.62 

9.37 

26 

195.97 

9.76 

29 

Low at-risk b 


200.37 

5.94 

30 

204.17 

6.44 

24 

Combined at-risk 


197.70 

8.18 

56 

199.68 

9.31 

53 

Adjusted Posttest 








High at-risk a 


198.09 



200.27 



Low at-risk b 


197.33 



201.16 



Combined at-risk 


197.71 



200.71 
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Metacognitive Strategy Knowledge 
Pretest 


High at-risk a 

9.56 

1.63 

27 

9.11 

1.77 

28 

Low at-risk b 

8.87 

1.98 

31 

9.13 

1.48 

24 

Combined at-risk 

9.19 

1.84 

58 

9.22 

1.65 

55 

Posttest 







High at-risk a 

8.81 

1.65 

26 

10.03 

1.78 

29 

Low at-risk b 

9.19 

1.99 

31 

10.21 

1.79 

24 

Combined at-risk 

9.02 

1.84 

57 

10.20 

1.76 

56 

Adjusted Posttest 







High at-risk a 

8.44 



9.66 



Low at-risk b 

9.13 



10.10 



Combined at-risk 

8.78 



9.88 




Note: “scored < 26th percentile in mathematics; b scored > 26th percentile and <40 th percentile in 
mathematics; SBI = schema-based instruction; WPS = word problem solving; MAP = Measures 
of Academic Progress. 
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